Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add changes for e2e inference for passing test cases #1127

Merged
merged 1 commit into from
Feb 4, 2025

Conversation

kamalrajkannan78
Copy link
Contributor

@kamalrajkannan78 kamalrajkannan78 commented Jan 28, 2025

Summary

  • End to end inference changes added for passing test cases present in Mobilenet v1, Mobilenet v2, Mobilenet v3, Resnext, wideresnet , ghostnet & DLA Models
  • Push marker added for vilt & all above models

Copy link

TestsPassed ✅Skipped ⚠️Failed
TT-Forge-FE Tests505 ran442 passed63 skipped0 failed
TestResult
No test annotations available

Copy link

TestsPassed ✅Skipped ⚠️Failed
TT-Forge-FE Tests563 ran485 passed78 skipped0 failed
TestResult
No test annotations available

Comment on lines 64 to 74
# Post processing
probabilities = torch.nn.functional.softmax(output[0][0], dim=0)

url = "https://raw.githubusercontent.com/pytorch/hub/master/imagenet_classes.txt"
urllib.request.urlretrieve(url, "imagenet_classes.txt")

with open("imagenet_classes.txt", "r") as f:
categories = [s.strip() for s in f.readlines()]
top5_prob, top5_catid = torch.topk(probabilities, 5)
for i in range(top5_prob.size(0)):
print(categories[top5_catid[i]], top5_prob[i].item())
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I've noticed that this pattern arises in other models as well. Can we put this post processing part inside utils folder e.g. ghostnet/utils/utils.py where we would have method something like this:

def post_processing(top_k, url, file_name):
    # Post processing
    probabilities = torch.nn.functional.softmax(output[0][0], dim=0)
    urllib.request.urlretrieve(url, file_name)

    with open(file_name, "r") as f:
        categories = [s.strip() for s in f.readlines()]
    topk_prob, topk_catid = torch.topk(probabilities, top_k)
    for i in range(topk_prob.size(0)):
        print(categories[topkcatid[i]], topk_prob[i].item())

Also let's do this for other models as well :)

Comment on lines 23 to 43
# Create model
model = MobileNetV1(9)
model.eval()

# Load data sample
url, filename = ("https://github.com/pytorch/hub/raw/master/images/dog.jpg", "dog.jpg")
urllib.request.urlretrieve(url, filename)

# Preprocessing
input_image = Image.open(filename)
preprocess = transforms.Compose(
[
transforms.Resize(256),
transforms.CenterCrop(224),
transforms.ToTensor(),
transforms.Normalize(mean=[0.485, 0.456, 0.406], std=[0.229, 0.224, 0.225]),
]
)
input_tensor = preprocess(input_image)
input_batch = input_tensor.unsqueeze(0)

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Initialization of all mobilenet_vx models is almost the same, can we also put that function in .../mobilenet/utils/ folder so we can reuse it for other mobilenets as well.

@kamalrajkannan78 kamalrajkannan78 force-pushed the kkannan/add_post_processing_steps_jan28 branch 2 times, most recently from bd924fc to 13e9f5b Compare January 29, 2025 15:59
Copy link

TestsPassed ✅Skipped ⚠️Failed
TT-Forge-FE Tests505 ran441 passed64 skipped0 failed
TestResult
No test annotations available

Copy link

TestsPassed ✅Skipped ⚠️Failed
TT-Forge-FE Tests564 ran487 passed77 skipped0 failed
TestResult
No test annotations available

Comment on lines 35 to 47
def post_processing(output, url="https://raw.githubusercontent.com/pytorch/hub/master/imagenet_classes.txt"):

probabilities = torch.nn.functional.softmax(output[0][0], dim=0)
urllib.request.urlretrieve(url, "imagenet_classes.txt")

with open("imagenet_classes.txt", "r") as f:
categories = [s.strip() for s in f.readlines()]
top5_prob, top5_catid = torch.topk(probabilities, 5)
for i in range(top5_prob.size(0)):
print(categories[top5_catid[i]], top5_prob[i].item())
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Please pass the number of top k categories as a param to the function (where default is 5) because we might not always want the top 5 categories. Apply this change in other utils as well :)

@kamalrajkannan78 kamalrajkannan78 force-pushed the kkannan/add_post_processing_steps_jan28 branch 2 times, most recently from a81ea0c to a470a9f Compare January 31, 2025 16:43
Comment on lines 51 to 53
top5_prob, top5_catid = torch.topk(probabilities, top_k)
for i in range(top5_prob.size(0)):
print(categories[top5_catid[i]], top5_prob[i].item())
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Please rename the top5_prob and top5_catid to topk_prob and topk_catid in this and other files as well.

Copy link

TestsPassed ✅Skipped ⚠️Failed
TT-Forge-FE Tests568 ran489 passed79 skipped0 failed
TestResult
No test annotations available

Copy link

TestsPassed ☑️Skipped ⚠️Failed ❌️
TT-Forge-FE Tests510 ran448 passed59 skipped3 failed
TestResult
TT-Forge-FE Tests
pytest
test_mobilenet_v3.test_mobilenetv3_basic[mobilenet_v3_large]❌ failure
test_resnext.test_resnext_50_torchhub_pytorch[resnext50_32x4d]❌ failure
test_wideresnet.test_wideresnet_pytorch[wide_resnet50_2]❌ failure

@kamalrajkannan78 kamalrajkannan78 force-pushed the kkannan/add_post_processing_steps_jan28 branch 2 times, most recently from 8361b90 to 87f7c99 Compare February 2, 2025 18:44
Copy link

github-actions bot commented Feb 2, 2025

TestsPassed ✅Skipped ⚠️Failed
TT-Forge-FE Tests568 ran489 passed79 skipped0 failed
TestResult
No test annotations available

Copy link

github-actions bot commented Feb 2, 2025

TestsPassed ✅Skipped ⚠️Failed
TT-Forge-FE Tests510 ran451 passed59 skipped0 failed
TestResult
No test annotations available



@pytest.mark.nightly
@pytest.mark.parametrize("variant", variants, ids=variants)
@pytest.mark.parametrize("variant", params)
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pls rename back to variants to maintain the standard format



@pytest.mark.nightly
@pytest.mark.parametrize("variant", variants, ids=variants)
@pytest.mark.parametrize("variant", params)
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

same here too



variants = ["wide_resnet50_2", "wide_resnet101_2"]
params = [
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

same here

categories = [s.strip() for s in f.readlines()]
topk_prob, topk_catid = torch.topk(probabilities, top_k)
for i in range(topk_prob.size(0)):
print(categories[topk_catid[i]], topk_prob[i].item())
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Please cleanup downloaded imagenet_classes.txt file after successful post processing


# STEP 3: Prepare input
url, filename = ("https://github.com/pytorch/hub/raw/master/images/dog.jpg", "dog.jpg")
urllib.request.urlretrieve(url, filename)
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Please cleanup downloaded image file after successful processing. Also pls update for other models too

@kamalrajkannan78 kamalrajkannan78 force-pushed the kkannan/add_post_processing_steps_jan28 branch 2 times, most recently from 4cdb2b0 to 274a0f8 Compare February 3, 2025 06:13
Copy link

github-actions bot commented Feb 3, 2025

TestsPassed ✅Skipped ⚠️Failed
TT-Forge-FE Tests510 ran451 passed59 skipped0 failed
TestResult
No test annotations available

Copy link

github-actions bot commented Feb 3, 2025

TestsPassed ✅Skipped ⚠️Failed
TT-Forge-FE Tests568 ran489 passed79 skipped0 failed
TestResult
No test annotations available

@kamalrajkannan78 kamalrajkannan78 force-pushed the kkannan/add_post_processing_steps_jan28 branch 2 times, most recently from eb45ecf to 0d602c6 Compare February 3, 2025 18:23
@kamalrajkannan78 kamalrajkannan78 force-pushed the kkannan/add_post_processing_steps_jan28 branch from 0d602c6 to eb4f134 Compare February 3, 2025 19:39
Copy link

github-actions bot commented Feb 3, 2025

TestsPassed ✅Skipped ⚠️Failed
TT-Forge-FE Tests510 ran451 passed59 skipped0 failed
TestResult
No test annotations available

1 similar comment
Copy link

github-actions bot commented Feb 3, 2025

TestsPassed ✅Skipped ⚠️Failed
TT-Forge-FE Tests510 ran451 passed59 skipped0 failed
TestResult
No test annotations available

Copy link

github-actions bot commented Feb 3, 2025

TestsPassed ✅Skipped ⚠️Failed
TT-Forge-FE Tests568 ran489 passed79 skipped0 failed
TestResult
No test annotations available

1 similar comment
Copy link

github-actions bot commented Feb 3, 2025

TestsPassed ✅Skipped ⚠️Failed
TT-Forge-FE Tests568 ran489 passed79 skipped0 failed
TestResult
No test annotations available

@kamalrajkannan78 kamalrajkannan78 merged commit ccb3f6b into main Feb 4, 2025
10 checks passed
@kamalrajkannan78 kamalrajkannan78 deleted the kkannan/add_post_processing_steps_jan28 branch February 4, 2025 06:33
vwellsTT pushed a commit that referenced this pull request Feb 4, 2025
### Summary

- End to end inference changes added for passing test cases present in
Mobilenet v1, Mobilenet v2, Mobilenet v3, Resnext, wideresnet , ghostnet
& DLA Models
- Push marker added for vilt & all above models
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants